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Abstract 

It is shown that distributions arising in Renyi-Tsallis max- 
imum entropy setting are related to the Generalized Pareto 
Distributions (GPD) that are widely used for modeling the 
tails of distributions. The relevance of such modelization, as 
well as the ubiquity of GPD in practical situations follows 
from Balkema-De Haan-Pickands theorem on the distribution 
of excesses (over a high threshold). We provide an entropic 
view of this result, by showing that the distribution of a 
suitably normalized excess variable converges to the solution 
of a maximum Tsallis entropy, which is the GPD. This result 
resembles the entropic approach to the Central Limit theorem 
as provided in [ 1 1 ; however, the convergence in entropy proved 
here is weaker than the convergence in supremum norm given 
by Pickands' theorem. 

I. Introduction 

Generalized Pareto Distributions (GPD) are widely used in 
practice for modeling the tails of distributions. The underlying 
rationale is the Balkema-De Haan-Pickands theorem 0, Q, 
which asserts that the distribution function of the excess 
variable X ~ u\X > u (i.e. the distribution of the shifted 
variable X exceeding a threshold u) converges, as u — > oo, to 
a GPD with survival function: 

S x (x) = Pr(X >x) = \ (1) 

where a is a scale parameter and 7 a shape parameter; for 7 = 
0, the GPD reduces to the exponential distribution Sx(x) = 
exp (— x/cr). The corresponding density is 

<7 \ a J 

for 7^0, and reduces to fx{x) — l/<rexp(— x/a) for 7 = 0. 

In applied fields, GPD have encountered a large success 
since they were obtained as the maximizers of a special 
entropy, the Tsallis (Havrda-Charvat-Darovczy) entropy (4), 
with suitable constraints. This entropy is defined by 

H g (fx) = Y^(J fx (*) dx - (2) 
for q > 0. We note that Shannon entropy 

Hi (fx) = lim H q = - [ f x (x) log f x (x)dx (3) 
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Figure 1. Infinite support Generalized Pareto Densities for several values of 
the parameter 7, with a = 1, 



is recovered in the limit case q = 1. 

It is worth mentioning that any monotonous transform of 
the latter entropy exhibits the same GPD maximizers: an 
important example is Renyi entropy Q. The GPD distribution 
is of very high interest in many physical systems, since it 
enables to model power-law phenomena. Indeed, power-laws 
are especially interesting since they appear widely in physics, 
biology, economy, and many other fields j6). 

In this communication, we give an interpretation of 
Pickands' theorem which relates it to the maximum 
(Renyi/Tsallis) entropy setting; this view gives a possible 
interpretation for the ubiquity of 'Tsallis' (GPD) distributions 
in physics applications, as well as in other fields, an an 
argument in support to the use of Renyi/Tsallis entropies. 

In the following, we deal with univariate distributions 
defined on R or on a subset of R. Our approach is as 
follows: first, we show that the GPD can be obtained as the 
solution of a maximum Renyi-Tsallis entropy problem with 
proper normalization and moment constraints. Second, we 
consider distributions in the Frechet domain of attraction of 
distributions: this family includes for instance Cauchy, Student 
and Pareto distributions. We characterize the associated q- 
norm and first moment of the survival function associated 



to the excess variable X — u\X > u. Using an appropriate 
normalization, we define a variable whose survival function's 
g-norm and moment converge to constant values. We perform 
the same analysis for a subset of distributions in the domain 
of attraction of the Gumbel distributions. Third, we show that 
the distribution of excesses coincides asymptotically with the 
maximum Tsallis entropy solution. 

II. Solution to the maximization of Tsallis' 

ENTROPY 

We first derive the expression of the solution to the max- 
imization of Tsallis' entropy subject to normalization and 
moment constraints. 

Proposition 1: Consider the set T = {G : M+ -> K} . The 
maximum Tsallis entropy problem (or equivalently the maxi- 
mum g-norm problem), with q < 1, defined by 

max H a (G) 



+00 +00 

subject to J zG[z)dz = /1 and J G(z)dz = 9 



(4) 



has for unique solution 

G*(z)=a^ (l + ^zY 1 for q^l (5) 
where a > and j3 > 0. Moreover, for 1/2 < q < 10 

a.1- 1 (1 — q) 



1 -n 2 
(q — I) a 9-1 

<Z(2<7-1) a 2 



Q a \\n 119 a^- 1 ' (l-g) 

^"^"^ (2q-lY 



(6) 
(7) 



In the case q = 1, the unique solution writes and 

G.(z) = aexp(-Px) (8) 
with constants a and (3 such that 

rv 

(9) 



a 

w 



a 




and the Shannon entropy is 

fli(G.) - 



log a + a 



Proof: The solution of the maximum Shannon (q = 1) 
entropy problem is well documented. We only consider here 
the q < 1 case and we follow the approach of |7|. Consider 
the functional Bregman divergence: 

B(f,g) = J d(f,g)dx 

= - I ' {f{xY-g{xY)g{xf- x dx (10) 
+ q I (f(x)-g(x))g(xy- 1 dx 



associated to the (pointwise) Bregman divergence d(f, g) built 
upon the strictly convex function — x q for q e (0, 1). Then 
let us evaluate the divergence between the distribution G* (z) 
in © and any distribution G(z), with G dominated by G*, 
G(z) < and satisfying ©: 



B(G, G») = — / (G(z)'?-G f (z)'?) 
Js 



dz 



a / (G(z)G»(z) 9 - 1 -G > (z) 9 )d2 
Js 



G(z) q dz+ / G4z) q dz, (11) 

where 5 denotes the support of G*(z). The last line follows 
from the fact that since G and G* both satisfy (|4j, then, using 
(0 it is easy to check that 

G{x)G*{x) q - l dx= [ G*{x) q dx. 
s Js 

The Bregman divergence B(G, G») being always positive and 
equal to zero if and only if G = G», the equality (fTTT i implies 
that, for q € [0, 1[, 



ff g (G*) > ff 9 (G) 



(12) 



which means that G* is the distribution with maximum Renyi- 
Tsallis entropy, with q £ [0, 1[, in the set of all distributions 
G <C G* satisfying the constraints (@). Values of the con- 
straints (O and of the maximum entropy (|7]i follow by direct 
calculation. ■ 

III. The distribution of excesses for distributions 
in Frechet domain of attraction 

In the following, we consider the Frechet domain of attrac- 
tion: this is the set T of distributions such that if variables Xi 
are independent and identically distributed according to one of 
them, then maxi = i n {JQ} converges to the GPD distribution 
as n — > 00. It was shown by Gnedenko |8| that a necessary 
and sufficient condition for a distribution to be in the Frechet 
domain of attraction is that its survival function S(z) satisfies 



lim 



S(z) 
+00 S(cz) 



for all c > and for some tail index a > 0. Equivalently, this 
reads 

S(z) = z- a l(z), 

where l(z) is a slowly varying function, i.e. a function such 
that lim^ +oc ^ = 1, Vt > 0. 

Let us consider the excess variable X u = X — u\X > u. 
Its survival function is 



S x {z + u) 
Sx(u) ' 



*Note that the mean is not defined for q < 1/2 



Proposition 2: Suppose that X belongs to the Frechet do- 
main, with 

S x (z)~ z~ a l(z), 



then Sx n has asymptotic q— norm 



IS 



aq — 1 



and asymptotic first moment, with a > 2, 

+oo u 2 

zS Xu 0) dz = — r. 

o (l-a)(2-a) 

Proof: the q— power of the q— norm writes 



WSxAl 



+°° 'S x (z + u)^ g 



-oo 



o \ S x (w) 

fSx(z)\\ 

dz = u 



dz 



-oo 



fSx( 



urn, 



\Sx {u)J ' J i \ Sx (u) 

r+0 ° (uw)- aq , [ + °° 

u / i dw = u / w q dw 

li u- a J x 



dw 



u " Ji aq — 1 

with 1 — aq < 0, since a > 2,q > 1/2. Of course, we 
immediately obtain, taking 5=1, that 

WSxJx " 



a- 1 



Similarly, the first moment is 



zSx„ (z) ^z 

-/n 

+°° S X (z) , 

(z — uj — — -r^dz 

+ OC 



S x (z + u) 
z————dz 
b x (u) 

S x {wu) 
u (w — 1) — — ——uaw 
bx (u) 



(w — 1) w a dw 



(l-o) (2 -a)' 



We have a simple corollary to this result: 

Corollary 1: The survival function Sy u of random variable 
Y = Xj g (u) , where function g is such that g (u) ~ u, has 
asymptotic norms 

HSyJI, ~ (^—) 1/q and \\SyJ, = 

ag — 1 a — 1 

and an asymptotic first moment 

-+oo 

zSy u (z) ciz 



io (l-o)(2-o) 
Proof: The results for Sy u follow directly from Proposi- 
tion |2j with 

Sr u (z) = S JC „(^(n)). (13) 



IV. The distribution of excesses for a subset of 

DISTRIBUTIONS IN GUMBEL DOMAIN OF ATTRACTION 

For distributions in the Gumbel domain of attraction, the 
maximum of a set of variables converges to the Gumbel 
extreme value distribution. These distributions are character- 
ized by an "exponential" fall-off, and are said "light tailed" 
distributions. Their excesses over a threshold are exponentially 
distributed, which corresponds to a GPD with 7 = 0. The 
general characterization of the Gumbel domain of attraction 
involves an inconvenient condition on the derivative of the 



hazard function. We consider here only the Weibull subset W 
of the Gumbel domain of attraction, whose survival functions 
verify 

S(z) ~ exp (-z^l(z)) (14) 

where £ is the tail index and l(z) is a slowly varying function. 
This set contains for example the Gaussian and Gamma 
distributions: 

• The survival function of a Gaussian distribution with zero 
mean and unitary variance is Sx(%) = 1/2 erfc(x/\/2), 
with erfc the complementary error function. For x — > 
+00, the complementary error function is equivalent to 

erfc(x) ~ exp(— x 2 )/(xy/n), and 



Sx(x) 



1 



1 



l+2i 



V27T X 



'2tt 



The survival function of a Gamma distribution with shape 
parameter a and rate parameter b is given by Sx(x) = 
r(o, bx) /r(o). Since T(a, bx) - {bx) a ~ l e- bx , we obtain 



Sx{x) 



1 



r(a) 



Proposition 3: Suppose that X belongs to W, with 

S x (z) ~ expf-z^Z (z)), 

then the survival function Sx u of the excesses of X has 
asymptotic Shannon entropy 

Hi (Sx u , 



H{u) 

and asymptotic first moment and 1— norm 



+00 



zSx u (z) dz 



Sx u (z) dz 



u 1 ^ 



Proof: The computations are essentially the same as in 
the proof of Proposition [2] 

■ 

From this result, we deduce the 

Corollary 2: The survival function Sy u of random variable 
Y = u^~ 1 l(u)X has asymptotic Shannon entropy 

and asymptotic first moment and 1— norm 

zSy (z)dz~—Tr, / Sy (z)dz^- 
£ Jo ? 

V. The entropy solution and the distribution of 

EXCESSES 

Let us now show that the distributions of excesses, both in 
the Gumbel and the Frechet case, coincide with the maximum 
Tsallis entropy solution. 

In the Frechet domain of attraction, this result reads as 
follows 



Theorem 1: if X belongs to the Frechet domain of attrac- 
tion, then choosing q < 1 such that 

1 



1-9 



the distribution of the excesses Y u as defined in Corollary Q] 
reaches asymptotically the maximum q— norm solution under 
constraints asymptotically equal to p, and 9 provided a — (3 — 
1. 

Proof: Choosing a = yields 

1 1-q 



and 



I Si 



zSy u (z) dz 



aq — 1 
1 

" a- 1 " 
1 



2q-l' 
1-3 



(« -if 



lo (l-a)(2-a) q(2q-l) 

which coincide with the unique maximum q— norm function 
with constraints /i and 8 if and only if a = /3 = 1. 

Since the maximum entropy solution with the same con- 
straints is unique, we obtain that the excess variable from a 
distribution in the domain of attraction of Frechet distribution 
asymptotically follows a Generalized Pareto Distribution. ■ 

In the Gumbel case, we obtain similarly 
Theorem 2: if X belongs to W with 

Sx(z) ~ exp(— zH(z)) 

then the distribution of the excesses Y u as defined in Corollary 
preaches asymptotically the maximum Shannon entropy solu- 
tion under constraints asymptotically equal to // and 8 provided 
a = 1 and (3 — £. 

Proof: Equating the Shannon entropy and the first mo- 
ment and 1— norm of the maximum entropy solution of 
Proposition Q] with the same quantities reached asymptotically 
by Sy u as in Corollary [2] yields a = 1 and j3 = £. ■ 



Example 1: As an illustration, let us consider the Cauchy 
case. The pdf is given by 

2 

fx ( x ) = —FT, — 2T> x - °- 
7r (1 + or) 

Its survival function is (x) = 1 — - arctan (a;). Using now 
the fact that arctan(x) « iz/2—l/x, for x ^> 1, we obtain that 
Sx{%) ~ 2/(7rx), which means that the Cauchy distribution 
is in the Frechet domain of attraction, with exponent a = 1. 
The survival function of the excess variable X,, writes 



S 



Sx(x + u) 1 — f arctan (x + it) 



Sx(u) 1 — - arctan (u) 

Using the arctan(x) approximation again, we readily obtain, 
with the threshold !t> 1, 

1 



U + X u 



/- = (! + -) 



which has the form of the Generalized Pareto Distribution (Q]i 
with index 7=1. Finally, with Sy u (x) = Sx u (ux), we have 
%(x)~l/(l + x). 

Since we want to emphasize on the resemblance with the 
Central limit theorem, we mention the following stability 
property of the GPD ([T): the distribution of the excesses over a 
threshold of GPD remains a GPD, with the same exponent but 
a different shape parameter. This property is to be compared 
with the usual stability by addition of independent Gaussian 
random variables. 

Theorem 3: Given a GPD with parameters 7, a the dis- 
tribution of excesses remains a GPD with parameters 7 and 
a>=a(l + lu). 

Proof: As usual, let u denotes the threshold, Sx the 
survival function of the original GPD and Sx u the survival 
function of variable X u .Then, 



Sx u (x) 



S x (x + u) _ (l + gQc-m)) I 
Sx(u) 



1 



7 



with a' = a (l + %u) . ■ 

Note that in the limit 7 = case, the exponential distribution 
is invariant by thresholding, i.e. a' = a. 

VI. Final comments 

In the Frechet domain of attraction as well as in a subset of 
the Gumbel domain of attraction, we have connected the solu- 
tion of a maximum g-entropy (or maximum g-norm) problem 
with the asymptotic distribution of excesses over a threshold, 
and showed that these distributions are Generalized Pareto 
Distributions. With this result, it is possible to connect the 
ubiquity of heavy-tailed distributions in physics, economics 
or signal processing, the distribution of the excesses over a 
threshold, and a maximum entropy construction. 

Our approach shows the convergence in entropy of the dis- 
tributions of excess over a threshold; this type of convergence 
is in fact weaker than the distribution in supremum norm 
proved in Pickands' theorem. However, this work underlines 
an interesting parallel with the entropic proof of the Central 
Limit Theorem as given in (TJ. 
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